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Abstract. This article proposes novel off-line test generation techniques from non-deter- 
ministic timed automata with inputs and outputs (TAIOs) in the formal framework of 
the tioco conformance theory. In this context, a first problem is the determinization 
of TAIOs, which is necessary to foresee next enabled actions after an observable trace, 
but is in general impossible because not all timed automata are determinizable. This 
problem is solved thanks to an approximate determinization using a game approach. The 
algorithm performs an io-abstraction which preserves the tioco conformance relation and 
thus guarantees the soundness of generated test cases. A second problem is the selection 
of test cases from a TAIO specification. The selection here relies on a precise description 
of timed behaviors to be tested which is carried out by expressive test purposes modeled 
by a generalization of TAIOs. Finally, an algorithm is described which generates test cases 
in the form of TAIOs equipped with verdicts, using a symbolic co-reachability analysis 
guided by the test purpose. Properties of test cases are then analyzed with respect to the 
precision of the approximate determinization; when determinization is exact, which is the 
case on known determinizable classes, in addition to soundness, properties characterizing 
the adequacy of test cases verdicts are also guaranteed. 
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Introduction 

Conformance testing is the process of testing whether some implementation of a software 
system behaves correctly with respect to its specification. In this testing framework, imple- 
mentations are considered as black boxes, i.e. the source code is unknown, only their interface 
with the environment is known and used to interact with the tester. In formal model-based 
conformance testing, models are used to describe testing artifacts (specifications, implemen- 
tations, test cases, ...). Moreover, conformance is formally defined as a relation between 
implementations and specifications which reflects what are the correct behaviors of the im- 
plementation with respect to those of the specification. Defining such a relation requires the 
hypothesis that the implementation behaves as a model. Test cases with verdicts, which will 
be executed against the implementation in order to check conformance, are generated auto- 
matically from the specification. Test generation algorithms should then ensure properties 
relating verdicts of executions of test cases with the conformance relation {e.g. soundness), 
thus improving the quality of testing compared to manual writing of test cases. 

For timed systems, model-based conformance testing has already been explored in the 
last decade, with different models and conformance relations (see e.g. |22j for a survey), and 
various test generation algorithms (e.g. |H1 dH EI] ) • In this context, a very popular model 
is timed automata with inputs and outputs (TAIOs), a variant of timed automata (TAs) [Ij, 
in which the alphabet of observable actions is partitioned into inputs and outputs. We 
consider here a very general model, partially observable and non-deterministic TAIOs with 
invariants for the modeling of urgency. We resort to the tioco conformance relation de- 
fined for TAIOs [T7], which is equivalent to the rtioco relation [19]. This relation compares 
the observable behaviors of timed systems, made of inputs, outputs and delays, restricting 
attention to what happens after specification traces. Intuitively, an implementation con- 
forms to a specification if after any observable trace of the specification, outputs and delays 
observed on the implementation after this trace should be allowed by the specification. 

One of the main difficulties encountered in test generation for those partially observable, 
non-deterministic TAIOs is determinization. In fact determinization is required in order to 
foresee the next enabled actions during execution, and thus to emit a correct verdict de- 
pending on whether actions observed on the implementation are allowed by the specification 
model after the current observable behavior. Unfortunately, TAs (and thus TAIOs) are not 
determinizable in general [1]: the class of deterministic TAs is a strict subclass of TAs. Two 
different approaches have been taken for test generation from timed models, which induce 
different treatments of non-determinism. 

• In off-line test generation test cases are first generated as timed automata (or timed 
sequences, or timed transition systems) and subsequently executed on the implemen- 
tation. One advantage is that test cases can be stored and further used e.g. for regres- 
sion testing and serve for documentation. However, due to the non-determinizability 
of TAIOs, the approach has often been limited to deterministic or determinizable 
TAIOs (see e.g. [151 IZI])- A notable exception is [T^ where the problem is solved 
by the use of an over-approximate determinization with fixed resources (number 
of clocks and maximal constant): a deterministic automaton with those resources 
is built, which simulates the behaviors of the non-deterministic one. Another one 
is [To] where winning strategies of timed games are used as test cases. 

• In on-line test generation, test cases are generated during their execution. After 
the current observed trace, enabled actions after this trace are computed from the 
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specification model and, either an allowed input is sent to the implementation, or 
a received output or an observed delay is checked. This technique can be applied 
to any TAIO, as possible observable actions are computed only along the current 
finite execution (the set of possible states of the specification model after a finite 
trace, and their enabled actions are finitely representable and computable), thus 
avoiding a complete determinization. On-line test generation is of particular interest 
to rapidly discover errors, can be applied to large and non-deterministic systems, 
but may sometimes be impracticable due to a lack of reactivity (the time needed 
to compute successor states on-line may sometimes be incompatible with real-time 
constraints). 

Our feeling is that off-line test generation from timed models did not receive much attention 
because of the inherent difficulty of determinization. However, recent works on approximate 
determinization of timed automata \18\ [7] open the way to new research approaches and 
results in this domain. 

Contribution. In this paper, we propose to generate test cases off-line for the whole class 
of non-deterministic TAIOs, in the formal context of the tioco conformance theory. The 
determinization problem is tackled thanks to an approximate determinization with fixed 
resources in the spirit of jl8j . using a game approach allowing to more closely simulate 
the non-deterministic TAIO [7j. Our approximate determinization method is more precise 
than jTH] (see [TIE] for details), preserves the richness of our model by dealing with partial 
observability and urgency, and can be adapted to testing by a different treatment of inputs, 
outputs and delays. Determinization is exact for known classes of determinizable TAIOs 
{e.g. event-clock TAs, TAs with integer resets, strongly non-Zeno TAs) if resources are 
sufficient. In the general case, determinization may over-approximate outputs and delays 
and under-approximate inputs. More precisely, it produces a deterministic io-abstraction 
of the TAIO for a particular io-refinement relation which generalizes the one of [9]. As a 
consequence, if test cases are generated from the io-abstract deterministic TAIO and are 
sound for this TAIO, they are guaranteed to be sound for the original (io-refined) non- 
deterministic TAIO. 

Behaviors of specifications to be tested are identified by means of test purposes. Test 
purposes are often used in testing practice, and are particularly useful when one wants 
to focus testing on particular behaviors, e.g. corresponding to requirements or suspected 
behaviors of the implementation. In this paper they are defined as open timed automata 
with inputs and outputs (OTAIOs), a model generalizing TAIOs, allowing to precisely target 
some behaviors according to actions and clocks of the specification as well as proper clocks. 
Then, in the same spirit as for the TGV tool in the untimed case [13], test selection is 
performed by a construction relying on a co-reachability analysis. Produced test cases are 
in the form of TAIOs, while most approaches generate less elaborated test cases in the form 
of timed traces or trees. In addition to soundness, when determinization is exact, we also 
prove an exhaustiveness property, and two other properties on the adequacy of test case 
verdicts. To our knowledge, this whole work constitutes the most general and advanced 
off-line test selection approach for TAIOs. 

This article is a long version of [_5j. In addition to the proofs of key properties, it also 
contains much more details, explanations, illustrations by examples, complexity considera- 
tions, and a new result on exhaustiveness of the test generation method. 
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Outline. The paper is structured as follows. In the next section we introduce the model 
of OTAIOs, its semantics, some notations and operations on this model and the model of 
TAIOs. Section 2 recalls the tioco conformance theory for TAIOs, including properties of 
test cases relating conformance and verdicts, and introduces an io- refinement relation which 
preserves tioco. Section 3 presents our game approach for the approximate determinization 
compatible with the io-refinement. In Section 4 we detail the test selection mechanism using 
test purposes and prove some properties on generated test cases. Section 5 discusses some 
issues related to test case execution and test purposes and some related work. 

1. A MODEL OF OPEN TIMED AUTOMATA WITH INPUTS/OUTPUTS 

Timed automata (TAs) [Ij is a usual model for time constrained systems. In the context of 
model-based testing, TAs have been extended to timed automata with inputs and outputs 
(TAIOs) whose sets of actions are partitioned into inputs, outputs and unobservable actions. 
In this section, we further extend TAIOs by partitioning the set of clocks into proper 
clocks {i.e., controlled by the automaton) and observed clocks (i.e, owned by some other 
automaton). The resulting model of open timed automata with inputs/outputs (OTAIOs for 
short), allows one to describe observer timed automata that can test clock values from other 
automata. While the sub-model of TAIOs (with only proper clocks) is sufficient for most 
testing artifacts (specifications, implementations, test cases) observed clocks of OTAIOs 
will be useful to express test purposes whose aim is to focus on the timed behaviors of the 
specification. Like in the seminal paper for TAs fl], we consider OTAIOs and TAIOs with 
location invariants to model urgency. 

1.1. Timed automata with inputs/outputs. We start by introducing notations and 
useful definitions concerning TAIOs and OTAIOs. 

Given X a finite set of clocks, a clock valuation is a mapping v : X ^ IR>0 5 where M>o 
is the set of non-negative real numbers. stands for the valuation assigning to all clocks. 
If u is a valuation over X and t £ M>o, then v + t denotes the valuation which assigns to 
every clock x £ X the value v{x) + 1. For X' C X we write v^x'^o] for the valuation equal 
to f on X \ X' and assigning to all clocks of X' . Given M a non- negative integer, an 
M -bounded guard (or simply guard) over X is a finite conjunction of constraints of the form 
X ~ c where x £ X, c £ [0, M] flN and ~€ {<,<,=,>,>}. Given g a guard and v a 
valuation, we write v \= g if v satisfies g. We sometimes abuse notations and write g for the 
set of valuations satisfying g. Invariants are restricted cases of guards: given M G N, an 
M -bounded invariant over X is a finite conjunction of constraints of the form x <\ c where 
X £ X,c £ [0,M] nN and < G {<,<}• We denote by Gm{X) (resp. Im{X)) the set of 
M-bounded guards (resp. invariants) over X. 

In the sequel, we write U for the disjoint union of sets, and use it, when appropriate, 
to insist on the fact that sets are disjoint. 

Definition 1.1 (OTAIO). An open timed automaton with inputs and outputs (OTAIO) is 
a tuple A = {L^, Ef, ^f, X^, X^, M^, I^, E^) such that: 

• L-^ is a finite set of locations, with £q £ L-^ the initial location, 

• Yjif, T,f and S:^ are disjoint finite alphabets of input actions (noted a?, 6?, . . .), output 
actions (noted a\,b\,...), and internal actions (noted ri,T2,...). We note S;^^^ = 
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U Tif for the alphabet of observable actions, and S'^ = 'Sf U 'Sf U S:^ for the 
whole set of actions. 

• Xp and are disjoint finite sets of proper clocks and observed clocks, respectively. 
We note X-^ = X^ U X^ for the whole set of clocks. 

• M-^ G N is the maximal constant of A, and we will refer to (|X-^|,M-^) as the 
resources of A, 

• I-^ : L-^ — )• I]^ja{X-^) is a mapping which labels each location with an M -bounded 
invariant, 

• E-^ C L"^ X G]^ja{X-^) X S"^ X 2 p X L-^ is a finite set of edges where guards are 
defined on X'^, but resets are restricted to proper clocks in Xp. 

One of the reasons for introducing the OTAIO model is to have a uniform model (syntax 
and semantics) that will be next specialized for particular testing artifacts. In particular, an 
OTAIO with an empty set of observed clocks X;^ is a classical TAIO, and will be the model 
for specifications, implementations and test cases. The partition of actions reflects their 
roles in the testing context: the tester cannot observe internal actions, but controls inputs 
and observes outputs (and delays). The set of clocks is also partitioned into proper clocks, 
i.e. usual clocks controlled by the system itself through resets, as opposed to observed clocks 
referring to proper clocks of another OTAIO {e.g. modeling the system's environment). 
These cannot be reset to avoid intrusiveness, but synchronization with them in guards and 
invariants is allowed. This partition of clocks will be useful for test purposes which can 
have, as observed clocks, some proper clocks of specifications, with the aim of selecting time 
constrained behaviors of specifications to be tested. 



X = 1, r, {x} 




Figure 1: Specification A 



Example 1.2. Figure^represents a TAIO for a specification A that will serve as a running 
example in this paper. Its clocks are X = Xp = {x}, its maximal constant is M'^ = 2, it 
has a single input = {a}, a single output T,f = {b} and one internal action T^i^ = {r}. 
Informally, its behavior is as follows. It may stay in the initial location Iq while x <1, and 
at X = 1, has the choice, either to go to ii with action r, or go to £5 with action t while 
resetting x. In ii, it may receive a and move to £2 when x is between 1 and 2, and reset x. 
In £2 it may stay while x < 1 and, either send b and go to £3 at x = 0, or loop silently when 
X = 1 while resetting x. This means that b can be sent at any integer delay after entering 
£2- In £3 it may stay while a; < 1 and move to ^4 when sending b. In £^, one can move to 
£q before x = 1 by receiving a and resetting x. Due to invariants x = Q in £% and £7, the 
subsequent behavior consists in the immediate transmission of two b 's. 
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1.2. The semantics of OTAIOs. Let A = {L-^,£^,l^f,l^f,^^,X^,X^,M^,I^,E^) be 
an OTAIO. The semantics of .4 is a timed transition system = {S-^, Sq, T^, -^^) where 

• S-^ = L-^ X R^(J^ is the set of states i.e. pairs {£,v) consisting in a location and a 
valuation of clocks; 

• Sq = {iQ,0) G S'^ is the initial state; 

• r-^ = M>o U E-^ X is the set of transition labels consisting in either a delay S or 

a pair (e, X'^) formed by an edge e £ E and a set X'^ C X^ of observed clocks; 

• the transition relation — t-^C S'^ x F-^ x S-^ is the smallest set of the following moves: 

— Discrete moves: {i,v) ^^°a {f^' -.v') whenever there exists e = (i,g,a,Xp,£') G 
E-^ such that v \= g Al-^{£), X'^ C X^ is an arbitrary subset of observed clocks, 
v' = ■W[x^ux^-(-o] aiid v' \= Note that X'^ is unconstrained as observed 
clocks are not controlled by A but by a peer OTAIO. 

- Time elapse: {i, v) -A^ {£, v + 6) (or 5 € M>„ if v + 6 \= 

The semantics of OTAIOs generalizes the usual semantics of TAlOs. The difference 
lies in the treatment of the additional observed clocks as the evolution of those clocks is 
controlled by a peer OTAIO. The observed clocks evolve at the same speed as the proper 
clocks, thus continuous moves are simply extended to proper and observed clocks. For 
discrete moves however, resets of observed clocks are uncontrolled, thus all possible resets 
have to be considered. 

A partial run of ^ is a finite sequence of subsequent moves in {S-^ x r^)*.S-^. For 

example p = sq — -t^ Sg — si ■ ■ • Sk-i — — >a Sk- ihe sum of delays m p is 
noted time{p). A run is a partial run starting in Sq . A state s is reachable if there exists a 
run leading to s. A state s is co-reachable from a set S' C 5-^ if there is a partial run from 
s to a state in S' . We note reach(^) the set of reachable states and coreach(^, S') the set 
of states co-reachable from S' . 

A (partial) sequence is a projection of a (partial) run where states are forgotten, and 
discrete transitions are abstracted to actions and proper resets which are grouped with 
observed resets. As an example, the sequence corresponding to a run 

5, , (eiX) 4 , (efe,X„*) 
P — Sq ^A Sq >A Si - ■ ■ >A Sf^-l >A Sk 

is 

/X = <5i.(ai, U Xl) ■ ■ ■ Sk.iak, X^ U X^) 
where = {£i, gi,ai,Xp,£'j) for all i G [1,A;]. We then note sq — ^a Sk- We write sq — ^a if 

there exists Sk such that Sq -^a Sk- We note Seq(^) C (M>o u (E-^ x 2'^'^))* (respectively 
pSeq(^)) the set of sequences (resp. partial sequences) of A. For a sequence fi, time{p,) 
denotes the sum of delays in p,. 

For a (partial) sequence p G pSeq(^), Trace{p) G (M>oLJS^^^)*.M>o denotes the observ- 
able behavior obtained by erasing internal actions and summing delays between observable 
ones. It is defined inductively as follows: 

• Tracers) = 0, 

• Trace{5i ...6k) = ^i=iSi, 

• Trace{5i . . . Sk-{T, X').p) = Trace{{Ti^^^5,i).p), 

• Trace{6i . . . 5k.{a,X').p) = {^^^6i).a.Trace{p) if a G E^^^. 



OFF-LINE TEST SELECTION WITH TEST PURPOSES FOR NON-DET. TIMED AUTOMATA 



7 



For example Trace{l.{T, X^).2.{a, X'^).2.{t, X^)) = 3. a. 2 and Trace{l.{T, X^).2.{a, X^)) = 
3.a.0. When a trace ends by a 0-delay, we sometimes omit it and write e.g. 3. a for S.a.O. 

When concatenating two traces, the last delay of the first trace and the initial de- 
lay of the second one must be added up as follows: if ai = Si.ai. ■ ■ ■ an-Sn+i and (72 = 
5'i.a'i. ■ ■ ■ a^.5^_,_]^ then cri.a2 = 6i.ai. ■ ■ ■ o„.(5„+i + 6[).a'i. ■ ■ ■ a^.5^_j_]^. Concatenation al- 
lows one to define the notion of prefix. Given a trace a, ai is a prefix of a if there exists 
some (72 with a = (Ti.a2. Under this definition, l.a.l is a prefix of l.a.2.6. 

For a run p projecting onto a sequence n, we also write Trace{p)foT Trace{p). The set 
of traces of runs of is denoted by Traces(wA) C (]R>o U S^^^)*.]R>o I 

Two OTAIOs are said equivalent if they have the same sets of traces. 

Let a S (M>o U S^^g)*.M>o be a trace, and s S S''^ be a state, 

• A after o" = {s G 5'^ | 3/i E Seq(^), Sq — s A Trace{fi) = a} denotes the set of 
states where A can stay after observing the trace a. 

• elapse{s) = {t £ M>o | 3p G (M>o U (S^ x 2^"^))*, s -A^ Mime{p) = t} is the set 
of enabled delays in s with no observable action. 

• out{s) = {a G Tif I 3X C X-^, s -^^-1} U elapse{s) (and in{s) = {a G T,f \ s ^-^\}) 
for the set of outputs and delays (respectively inputs) that can be observed from s. 
For S' C S'^, out{S') = Usg5' out{s) and in{S') = Uses' ^'^(■s)- 

Using these last definitions, we will later describe the set of possible outputs and delays 
after the trace a by out{A after a). 

Notice that all notions introduced for OTAIOs apply to the subclass of TAIOs. 



1.3. Properties and operations. A TAIO A is deterministic (and called a DTAIO) when- 
ever for any a G Traces(^), ^ after o" is a sing letorfl A TAIO A is determinizable if there 
exists an equivalent DTAIO. It is well-known that some timed automata are not determiniz- 
able [1]; moreover, the determinizability of timed automata is an undecidable problem, even 
with fixed resources |24^ [T2] . 

An OTAIO A is said complete if in every location i, = true and for every action 

a G S"^, the disjunction of all guards of transitions leaving i and labeled by a is true. This 
entails that Seq(^) ix-^— ^>o U (^'^ ))*) where \^x^ is the projection that removes 

resets of proper clocks in X^. This means that A is universal for all the behaviors of its 
environment. 

An OTAIO A is input- complete in a state s G reach(^), if in{s) = An OTAIO A 
is input-complete if it is input-complete in all its reachable states. 

An OTAIO A is non-blocking if Vs G reach(^), Vt G M>o, 3/x G pSeq(^) n (M>o U ((Sf* U 

S:^) X 2^'^))* ,time{p) = t A s -^a- This means that it never blocks the evolution of time, 
waiting for an input. 

For modeling the behavior of composed systems, in particular for modeling the execution 
of test cases on implementations, we introduce the classical parallel product. This operation 
consists in the synchronization of two TAIOs on complementary observable actions {e.g. a!, 

^Notice that formally, a trace always ends with a delay, which can be 0. This technical detail is useful 
later to define verdicts as soon as possible without waiting for a hypothetical next action. 

^Determinism is only defined (and used in the sequel) for TAIOs. For OTAIOs, the right definition 
would consider the projection of A after o which forgets values of observed clocks, as these introduce 
"environmental" non-determinism. 
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the emission of a and a? its reception) and induces the intersection of the sets of traces. It 
is only defined for compatible TAIOs, i.e. A"- = (L'J'q, T.\, S|, S;, X'p, M\ P, E') for i = 1, 2 
such that = S^, = Sf , n = and n X ^ = 0.' 

Definition 1.3 (Parallel product). The parallel product of two compatible TAIOs A!' = 
{L%tQ,T,l,,T,\,T,l^,X'p,M',I',E') i = 1,2 is a TAIO A^A'^ = (L, 4, Sv, Si, S^, Xp, M, J, ^) 
where: 

• T.7 = S^, Si = Sf and S^ = U 

• = u 

• M = max(M\M2) 

• V(^i, F) G L, = A /(F) 

• is the smallest relation such that: 

- for a G U Sf, if {£\ g\ a, i") £ E' and {P , g\ a, X'p\ i'^) G then 
((£\ £^), 5^ A g'^, a, X^ U Xp, (£'\ f ^)) G .B, i.e. complementary actions synchro- 
nize, corresponding to a communication; 

- foTTieKJ' &L\if{e,g\n,X^\nGE' then{{e,e),g\n,X'p\{e\e)) G 
E, i.e. internal actions of progress independently; 

- forrs G ^l, t G L\ii{e,g\T2,X'p\e'') € EHhen g\T2,X'p\ G 
E, i.e. internal actions of ^2 progress independently. 

By the definition of the transition relation E of ^^||.4^, TAIOs synchronize exactly on 
complementary observable actions and time, and evolve independently on internal actions. 
As a consequence, the following equality on traces holds: 

Traces(^i^') = Traces(^') n Traces(^^) (1.1) 

Notice that the definition is not absolutely symmetrical, as the direction (input/output) 
of actions of the product is chosen with respect to A^. The technical reason is that, in the 
execution of a test case on an implementation, we will need to keep the directions of actions 
of the implementation. 



I A' I A = A^\\A^ I 



x; = {y 




Figure 2: Example of a parallel product .A = ^^||^^. 



Example 1.4. The Figure^ gives a very simple illustration of the parallel product. The 
intersection of the sets of traces is clear. Indeed, the parallel product recognizes exactly all 
prefixes of the trace l.a.1.6. 



OFF-LINE TEST SELECTION WITH TEST PURPOSES FOR NON-DET. TIMED AUTOMATA 



9 



We now define a product operation on OTAIOs which extends the classical product of 
TAs, with a particular attention to observed clocks. This product is used later in the paper, 
to model the action of a test purpose which observes the clocks of a specification. 

Definition 1.5 (Product). Let A' = (L% ^5, S?, Si, E^, X^, X^, M% /% ^'), i = 1,2, be two 
OTAIOs with same alphabets and disjoint sets of proper clocks {Xp n Xp = 0). Their 
product is the OTAIO x = {L,iQ,T.7,T.\,T.r,Xp,Xo,M,I ,E) where: 

• L = U xL^; 

. 4 = ^ 

• Xp = XpU Xp, Xo = {XI U XI) \ Xp-, 

• M = max(M\M2); 

• y{e\P) £ L,i{{e\P)) = p{£')aP{P); 

. iie,n,g'Ag\a,X';uX^\ G ii; if {t,g\a,X;;,i'^) E E% i=l,2. 

Intuitively, A^ and A^ synchronize on both time and common actions (including internal 
one^. A^ may observe proper clocks of A^ using its observed clocks Xp n X^, and vice 
versa. The set of proper clocks of A^ x A'^ is the union of proper clocks of A^ and A'^, and 
observed clocks of A^ x A'^ are observed clocks of any OTAIO which are not proper. For 
example, the OTAIO in Figure 13 represents the product of the TAIO A in Figure [T] and 



the OTAIO TV of Figure 12 



z = 1 /\y >1 Ax <1, ah {z} 



z<l/\y>lAx = 2.b]\ 



X; = {z}, Xl = {x,y} 



A' 



a: = 1. a?, 



y > 1,6! 



X'^ = {x}, Xl = {y,z} 



A = A^xA' I 

Q 

1 A y > 1 A 2; = l,a?,{x.z} 

o 

z < 1 /\y > 1 /\ X = 2,b\ 



X^ = {x.z}, X„ = {y} 



Figure 3: Example of a product A = A^ x A'. 



Contrary to the parallel product, the set of traces of the product of two OTAIOs is not 
the intersection of the sets of traces of these TAIOs, as illustrated by the following example. 

Example 1.6. Figure^ artificially illustrates the notion of product of two OTAIOs. One 
can see that l.a?.1.6! is a trace of A^ and A^ hut is not a trace of A = A^ x A^ . Indeed, in 
A^ , l.a?.1.6! is the trace of a sequence where x is not reset at the first action. Unfortunately, 
the clock X is observed by A^ but is a proper clock of A^ which resets it at the first action. As 
a consequence, l.a?.1.6! cannot be a trace of the product A^ xA^. In fact, the second edge in 
A can never be fired, since clocks z and x agree on their values and cannot he simultaneously 
smaller than 1 and equal to 2. 

On the other hand, sequences are more adapted to express the underlying operation. 
To compare the sets of sequences of A^ x A^ with the sets of sequences of its factors, we in- 
troduce an operation that lifts the sets of clocks of factors to the set of clocks of the product: 

Synchronizing internal actions allows for more precision in test selection. This justifies to have a set of 
internal actions in the TAIO model. 
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for defined on {Xp,X^), and Xp D X^ = 0, ^i|(^p'^2) denotes an automaton identical 
to but defined on (X^, X^ U X^ U XI \ Xp). The effect on the semantics is to duplicate 

moves of A^ with unconstrained resets in (XpUX^) \ (XpUXl), so that A^t^^^'^°'' strongly 
bisimulates A^. The equivalence just consists in ignoring values of added clocks which do 
not interfere in the guards. Similarly -g (iggj^gd on {X^,X^ Li X^Li X^\ X^). 

Both A't^p'^° and A'l^p'^' have sequences in (M>o U (S^ x {X^U X^U X'^U X^)))* . They 
synchronize on both delays and common actions with their resets. The effect of the product 
is to restrict the respective environments (observed clocks) by imposing the resets of the 
peer TAIO. The sequences of the product are then characterized by 

Seq(^^ X A') = Seq{A't^^^'^°^) n Seq(^'t^^^'^°^) (1-2) 
meaning that the product of OTAIOs is the adequate operation for intersecting sets of 
sequences. 

An OTAIO equipped with a set of states F C S"-^ can play the role of an acceptor. A 
run is accepted in F if it ends in F. Seq^(^) denotes the set of sequences of accepted runs 
and Tracesi7'(^) the set of their traces. By abuse of notation, if L is a subset of locations 
in L"^, we note Seqr (^) for Seq x-^i-^) and similarly for Traces^f^). Note that for the 

product A^ X A^, if F^ and F^ are subsets of states of A^ and A^ respectively, additionally 



to (1.2), the following equality holds: 

Seqpi^F^-^' X ■^') = Seq^i(^it^^^''^°^) n Seq^2 (^^t^^^'^°)). (1.3) 



2. Conformance testing theory 

In this section, we recall the conformance theory for timed automata based on the confor- 
mance relation tioco [18j that formally defines the set of correct implementations of a given 
TAIO specification, tioco is a natural extension of the ioco relation of Tretmans [23] to 
timed systems. We then define test cases, formalize their executions, verdicts and expected 
properties relating verdicts to conformance. Finally, we introduce a refinement relation 
between TAIOs that preserves tioco, and will be useful in proving test case properties. 

2.1. The tioco conformance theory. We consider that the specification is given as a 
(possibly non-deterministic) TAIO A. The implementation is a black box, unknown except 
for its alphabet of observable actions, which is the same as the one of A. As usual, in order 
to formally reason about conformance, we assume that the implementation can be modeled 
by an (unknown) TAIO. Formally: 

Definition 2.1 (Implementation). Let A = {L-^,£^,^f,^f,Y.:^,X^,$,M-^,I'^,E^) be a 
specification TAIO. An implementation of A is an input-complete and non-blocking TAIO 
I = (L^,£g,S7,S!,E^,X^,0,M^,F,E^) with same observable alphabet as A (Sf = 
and Tif = T,f). I{A) denotes the set of possible implementations of A. 

The requirements that an implementation is input-complete and non-blocking will en- 
sure that the execution of a test case on X does not block before verdicts are emitted. 

Among the possible implementations in I{A), the conformance relation tioco (for timed 
input- output conformance) |18] formally defines which ones conform to A, naturally extend- 
ing the classical ioco relation of Tretmans [23j to timed systems: 
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Definition 2.2 (Conformance relation). Let ^ be a TAIO representing the specification 
and X € I{A) be an implementation of A. We say that I conforms to A and write 
X tioco ^ if Vfj G Traces(^), outiX after a) C out{A after a). 

Note that tioco is equivalent to the rtioco relation that was defined independently 
in [T9j (see [22j). Intuitively, X conforms to A if after any timed trace enabled in A, every 
output or delay of X is specified in A. This means that X may accept more inputs than A, 
but is authorized to send less outputs, or send them during a more restricted time interval. 
The intuition is illustrated on the following simple example: 

Example 2.3. Figure\^ represents a specification A and two possible implementations X\ 
andX2. Note thatXi andX2 should be input- complete, but for simplicity of figures, we omit 
some inputs and consider that missing inputs loop to the current location. It is easy to see 
thatXi conforms to A. Indeed, it accepts more inputs, which is allowed (after the trace e, X\ 
can receive a and d while A only accepts a ), and emits the output b during a more restricted 
interval of time (outiXi after a. 2) = [0, oo) is included in out{A after a. 2) = [0, c«)U{6} ). 
On the other hand X2 does not conform to A for two reasons: X2 may send a new output 
c and may send b during a larger time interval (e.g. out(X2 after a.l) = [0, 00) U {b,c} is 
not included in out(A after a.l) = [0,oo)). 




Figure 4: Example of a specification A and two implementations Xi and X2. 

In practice, conformance is checked by test cases run on implementations. In our setting, 
we define test cases as deterministic TAIOs equipped with verdicts defined by a partition 
of states. 

Definition 2.4 (Test suite, test case). Given a specification TAIO A, a test suite is a set 
of test cases, where a test case is a pair (TC, Verdicts) consisting of: 

• a deterministic TAIO TC = (L^^ ^ ^ M^^,^^, E^^), 

• a partition Verdicts of the set of states S'^'' = None UlnconcU Pass U Fail. States 
outside None are called verdict states. 

We also require that 

• = and = ^7, 

• TC is non-blocking, {e.g. I^'^(^) = true for all £ G L'^^), 

• TC is input-complete in all None states, meaning that it is ready to receive any 
input from the implementation before reaching a verdict. 

In the following, for simplicity we will sometimes abuse notations and write TC for 
(T"C, Verdicts). Let us give some intuition about the different verdicts of test cases. Fail 
states are those where the test case rejects an implementation. The intention is thus to de- 
tect a non-conformance. Pass and Inconc states are linked to test purposes (see Section El): 
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the intention is that Pass states should be those where no non-conformance has been de- 
tected and the test purpose is satisfied, whereas Inconc states should be those states where 
no non-conformance has been detected, but the test purpose cannot be satisfied anymore. 
None states are all other states. We insist on the fact that those are intentional characteri- 
zations of the verdicts. Properties of test cases defined later specify whether these intentions 
are satisfied by test cases. We will see that it is not always the case for all properties. 

The execution of a test case TC G Test{A) on an implementation X G 1{A) is modeled 
by the parallel product X\\TC, which entails that Traces(X||TC) = Traces(X) n Traces(TC). 
The facts that TC is input-complete (in None states) and non-blocking while X is input- 
complete (in all states) and non-blocking ensure that no deadlock occurs before a verdict is 
reached. 

We say that the verdict of an execution of trace a E Traces(T'C), noted Verdict (a, T'C), 
is Pass, Fail, Inconc or None if TC after a is included in the corresponding states set [J 
We write X fails TC if some execution a of X||TC leads TC to a Fail state, i.e. when 
Tracespaii (TC) n Traces(X) / 0, which means that there exists a G Traces(X) n Traces(TC) 
such that Verdict(cr, TC) = Fail. Notice that this is only a possibility to reach the Fail 
verdict among the infinite set of executions of X||TC. Hitting one of these executions is not 
ensured both because of the lack of control of TC on X and of timing constraints imposed 
by these executions. 

We now introduce soundness, a crucial property ensured by our test generation method. 
We also introduce exhaustiveness and strictness that will be ensured when determinization 
is exact (see Section [4]). 

Definition 2.5 (Test suite soundness, exhaustiveness and strictness). A test suite TS for 
A is: 



Intuitively, soundness means that no conformant implementation can be rejected by the 
test suite, i.e. any failure of a test case during its execution characterizes a non-conformance. 
Conversely, exhaustiveness means that every non-conformant implementation may be re- 
jected by the test suite. Remember that the definition of X fails TC indicates only a 
possibility of reject. Finally, strictness means that non-conformance is detected once it oc- 
curs. In fact, -i(X||TC tioco A) means that there is a trace common to TC and X which 
does not conform to A. The universal quantification on X and TC implies that any such 
trace will fail TC. In particular, this implies that failure will be detected as soon as it 
occurs. 

Example 2.6. Figure represents a test suite composed of a single test case TC for the 
specification A of the Figure^ Indeed, TC is a TAIO which is input- complete in the None 
states. TS is sound because the Fail states of TC are reached only when a conformance 
error occurs, e.g. on trace l.b. However, this test case can observe non- conformant traces 
without detecting them, hence TS is not strict. For example, l.a.1.6, l.a.l.c and l.a.Q.c are 
non- conformant traces that do not imply a Fail verdict. These traces are e.g. traces 0/X2 



• sound if VX G I{A), \/TC € TS, X fails TC ^(X tioco A), 

• exhaustive if VX G I{A), ^(X tioco A) =^ 3TC G TS, I fails TC 

• strict if yi £l{A),yTC G TcS, ^(X||TC tioco ^) ^ X fails TC. 




which should allow to detect that -i(X2 tioco ^). 



Note that TC being deterministic, TC after a is a singleton. 
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TC 




None = {^1} X R>o U {£2} x [0, 8] 

Inconc = {^inc, ^2} X (8, 00) 

Pass = {£pass} X M>o 

Fail = {^Faii} X M>o 
TS = {TC} 



Figure 5: Example of a sound but not strict test suite for the specification A (Figure [4]). 

2.2. Refinement preserving tioco. We introduce an io-refinement relation between two 
TAIOs, a generalization to non-deterministic TAIOs of the io-refinement between DTAIOs 
introduced in [9] , itself a generalization of alternating simulation [2] . Informally A io- refines 
i3 if ^ specifies more inputs and allows less outputs and delays. As a consequence, if A 
and B are specifications, A is more restrictive than B with respect to conformance. We 
thus prove that io-abstraction (the inverse relation) preserves tioco: if X conforms to A, it 
also conforms to any io-abstraction B of A. This will ensure that soundness of test cases is 
preserved by the approximate determinization defined in Section [3j 

Definition 2.7. Let A and B be two TAIOs with same input and output alphabets, we say 
that A io-refines B (or B io-abstracts A) and note ^ ^ i3 if 

(i) Vcj G Traces(fi), out{A after a) C out{B after a) and, 

(a) Vo" G Traces(^), in{B after a) C in{A after a). 

As we will see below, ^ is a preorder relation. Moreover, as condition (ii) is always 
satisfied if A is input-complete, for X G T^A), X tioco A is equivalent to X ^ By 



transitivity of ^, it follows that io-refinement preserves conformance (see Proposition 2.9). 

Lemma 2.8. The io-refinement < is a preorder relation. 

Proof. The relation < is trivially reflexive and we prove that it is transitive. 
Suppose that A ^ B and B ^ C. By deflnition of ^ we have: 

V(T G Traces(;B), out{A after a) C out{B after a) (1) 

V(7 G Traces(^), in{B after a) C in{A after a) (2) and 

Vcj G Traces(C), out{B after a) C out{C after a) (3) 

Vct G Traces(;B), in{C after a) C in{B after a) (4) 

We want to prove that A ^ C thus that 

Vcr G Traces(C), out{A after a) C out{C after a) (5) 

Vcr G Traces(^), in{C after a) C in{A after a) (6) 

In order to prove (5), let a G Traces(C), and examine the two cases: 

• If cr G Traces(^)nTraces(C) then (1) and (3) imply out{A after a) C out{B after a) 
and out(B after a) C out{C after a). Thus out(A after a) C out(C after a) and 
we are done. 
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• If (J G Traces(C) \ Traces(^), there exist a', a" £ {T,obs U IR>o)* and a £ T,obs U ^>o 
such that a = a'.a.a" with a' e Traces(;S) fl Traces(C) and a'.a € Traces(C) \ 
Traces(e). As B ^ C, by (4) we get that a G U M>o. But as A ^ B, and 
fj' G Traces(;B), the condition (1) induces that out{A after a') C out{B after a'), 
and then a'.a G Traces(C) \ Traces(^). We deduce that out{A after a'.a) = 0, and 
thus out{A after o") = C out{C after a). 

The proof of (6) is similar. O 

Proposition 2.9. If A ^ B then VX G I{A) (= I{B)), I tioco ^ ^ X tioco B. 

Proof. This proposition is a direct consequence of the transitivity of ^. In fact when I is 
input-complete, by definition Vcj G Traces(X), in(X after o") = S?, thus condition (ii) of ^ 
trivially holds: Vcr G Traces(X), after a) C in{I after a). Thus Xtioco^ (which is 
defined by Vo" G Traces(^), out{X after a) C out{A after a)) is equivalent to X ^ Now 
suppose A^ B and X tioco ^ then the transitivity of < gives X tioco i3. D 




Figure 6: Counter-example to converse of Proposition 2.9 



Remark: unfortunately, the converse of Proposition 2.9 is in general false, already in the 
untimed case. This is illustrated in Figure [6| It is clear that the automaton A accepts all 
implementations. B also accepts all implementations as, from the conformance point of view, 
when a specification does not specify an input after a trace, this is equivalent to specifying 
this input and then to accept the universal language on Softs- Thus X tioco A^X tioco B. 
However -^{A ^ B) as in{B after e) = {o} but in{A after e) = 0. Notice that this example 
also works for the untimed case in the ioco conformance theory. 



As a corollary of Proposition 2.9, we get that io- refinement preserves soundness of test 
suites: 

Corollary 2.10. If A ^ B then any sound test suite for B is also sound for A. 

Proof. Let TS be a sound test suite for B. By definition, for any X G X{B), for any 



TC G TS, X fails TC =^ -i(X tioco B). As we have A ^ B, hy Proposition 2.9, we obtain 
-i(X tioco B) =^ -i(X tioco A) which implies that for any X G X{B), for any TC G TS, 
X fails TC =^ -i(X tioco A). Thus TS is also sound for A. □ 

In the sequel, this corollary will justify our methodology: from A a non-deterministic 
TAIO, build a deterministic io- abstraction B of A, then any test case generated from B and 
sound is also sound for A. 



3. Approximate determinization preserving conformance 

We recently proposed a game approach to determinize or provide a deterministic over- 
approximation for TAs [7] . Determinization is exact on all known classes of determinizable 
TAs {e.g. event-clock TAs, TAs with integer resets, strongly non-Zeno TAs) if resources 
(number and clocks and maximum constant) are sufficient. This method can be adapted to 
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the context of testing for building a deterministic io- abstraction of a given TAIO. Thanks 
to Proposition 2.9, the construction preserves tioco. 

The approximate determinization uses the classical regiorj^ construction [Ij. As for 
classical timed automata, the regions form a partition of valuations over a given set of clocks 
which allows to make abstractions in order to decide properties such as the reachability of 
a location. We note Reg(^x,M) the set of regions over clocks X with maximal constant M. 
A region r' is a time-successor of a region r if 3w S r, 3t £ M>o, v + t £ r' . Given X a set of 
clocks, a relation over X is a finite conjunction C of atomic constraints of the form x — y^c 
where x, y G X, ~G {<, =, >} and c G N. When all constants c belong to [— M, M] for some 
constant M G N we denote by Re\M{X) for the set of relations over X. Given a region r, 
we write for the smallest relation in RelM(^) containing r. 



3.1. A game approach to determinize timed automata. The technique presented 
in [7j applies first to TAs, z.e. the alphabet only consists of one kind of actions (say output 
actions), and the invariants are all trivial. Given such a TA A over set of clocks X^, a 
deterministic TA B with a new set of clocks X'^ is built, with Traces(^) = Traces(;B) as 
often as possible, or Traces(^) C Traces(i3). Resources of B are fixed, and the goal is to 
simulate the clocks of A by choosing the right resets in B. To this aim, letting k = \X'^\, 
a finite 2-player zero-sum turn-based safety game Qy^^i^k,M'^) ~ (^5) ^-D> vq, (^5 U d£),Bad) 
is built. The two players. Spoiler and Determinizator, alternate moves, the objective of 
player Determinizator being to remain in a set of safe states where intuitively, for sure no 
over-approximation has been performed. In this game, every strategy for Determinizator 
yields a deterministic automaton B with Traces(^) C Traces(i3), and every winning strategy 
induces a deterministic TA B equivalent to A. It is well known that for safety games, winning 
strategies can be chosen positional (i.e., only based on the current state) and computed in 
linear time in the size of the arena (see e.g. [2QJ). 

The game G^^(^k.M'^) — V/j, vq, (55 U 6d, Bad) is defined as follows: 
. V5 = 2^^""^"'— ("^>A^e)(^^ux8)x{±,T} ^ pjgg^^^^^^g^ -g ^j^g g^^^gg SpoHer. 

Each state is a pair vg = {£, r) where r is a region over X®, and iS is a finite set of 
configurations of the form {i, C, b) where £ is a location of C is a relation over 
X-^ U X'^ with respect to the maximal constant M = max(M'^, Af), and 6 is a 
boolean marker (T or _L). A state of Spoiler thus constitutes a state estimate of 
A, and the role of the marker b is to indicate whether over-approximations possibly 
happened. 

• Vd = V5 X (S X Reg(j(^8 jvfS)) is the set of states of Determinizator. Each state 
vd = (ys, {cL, 1"')) consists of a state of Spoiler, together with an action and a region 
over X'^ which role is to remember the last move of Spoiler. 

• vq = ({(^0) C'o, fto)}; {0}) G Vg, the initial state of the game, is a state of Spoiler 
consisting of a single configuration with the initial location £0 of A, the simple 
relation Co over X-^ U X'^: Vx,y G X-^ U X'^, x — y = 0, a marker bo = T (no 
over-approximation was done so far), together with the null region over X'^. 

• C V5 X (S X Reg(xe,A^B)) x V^) and 6d ^ x 2"^^ x V5 are inductively defined 
from Vq as follows: 



'Note that it could be adapted to zones with some loss in precision. 
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moves of Spoiler are pairs (a, r') and the successor of a state vg = {£, r) £ 
by the move (a,r') is simply v/5 = {{£ ,r), {a,r')), i.e. a copy of together 
with a challenge for Determinizator consisting in an action a and a region 
r' £ Reg(-;5cB^^e), a time-successor of r; 

moves of Determinizator are resets Y C X'^ and the successor of a state vd = 
{{£,r), (a,r')) G Vd by the reset Y C X'^, is the state of Spoiler (<S',r'y^Q|) G 

Vs where = {SucCe[{a,r'),Y]{e,C,b) \ {£,C,b) G £:} and 



SucCe[ia,r'),Y]{e,C, b) 



{£',C',b') 



3i i' (zE s.t. Ir' n C]\xA ng^ 
< >' 

M 



C = {r'nCng)[x^o][Y^o]' 

b' = bA{[r'nC]\xA^g) 

In words, £' is the set of elementary successors of configurations in £ by {a,r') 
and by resetting Y. An elementary successor of a configuration (i, C, b) by a 

transition £ ^'"'^> exists only if the guard [r'nC]|j^>i over X-^ induced by the 
guard r' over X'^ through the relation C intersects g. Intuitively, the transition 
is possible in i according to the state estimate (i, C) and the region r' . The 
resulting configuration (£',C",6') is such that: 

* / is the location reached by the transition; 

* C is the relation between clocks in X'^ and X^ after the moves of the two 
players, that is after satisfying the guard g m. r' r\ C, resetting X C X'^ 
and Y C 

* 6' is a boolean set to T if both b = T and the induced guard [r' n C]|x>i 
over X'^ implies g. Intuitively, b' becomes _L when r' encodes more values 
than g, thus an over-approximation possibly happens. 

Note that during the construction of 5s and 6d, the states of Determinizator whose 
successors by have an empty set of configurations are removed, together with the 
moves in 5s leading to them. Indeed these moves have no counterpart in A. 
• Bad = {{£,r) G V5 | y{i,C,b) G £,b = _L}. Bad states Determinizator wants to 
avoid are states where all configurations are marked _L, i.e. configurations where 
an approximation possibly happened. Note that a single configuration marked T 
in a state is enough to ensure that no over-approximation happened. Indeed, for 
any path in the game leading to such a state, starting from a T-marked config- 
uration, and taking elementary predecessors, one can build backwards a sequence 
of configurations following this path. By definition of the marker's update, these 
configurations are all marked T, and the sequence thus corresponds to real traces 
in the non-deterministic automaton. 

Example 3.1. Figure^ represents a simple non- deterministic timed automaton A. Let us 
explain how to construct the game ^^.(1,1) for A with resources (1,1), that is a single clock 
y and maximal constant 1. We only detail part of the construction in Figure\^ but the 
complete game can be found in [7\ . 

As defined above, the initial state of the game is simply vq = ({(^o, 2; — y = 0, T)}, {0}). 

From Vq, the only move of Spoiler compatible with behaviors of A is < y < l,a. 
Corresponding transitions in A lead to locations £0, t\ and I2, and only in this last location 
X has been reset. Each transition of A yields a configuration in the next state of Spoiler, 
and assuming Determinizator chooses to reset y, the three different configurations are the 
following: 
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Figure 7: Non- deterministic timed automaton A. 







io,x-y = 0,T {0} 


< y < l,a 





4,0 < x-y <1,T 
ei,0<x-y< 1,T 
(2,x- y = 0,T 



{0} 



<y <l,a 



{y} 



(0,0 < X - y < 1, 1 
li,0 < x - y < 1,1 
£2,-l<2;-y<0,± 



(0,1) 



Figure 8: Part of the game ^^4,(1,1)- 



• one with location ig, where x £ (0, 1) (no reset in A) and y = (reset in 

• one with location i\, where x G (0, 1) and y = 0, 

• and one with location £2, where x = (reset in A) and y = 0. 

In the two first configurations, the new relation is [y = < x < , that is < x — y < 1, 
and in the last configuration, the new relation is simply x — y = 0. As a consequence the 
successor state is vi = ({(^O) < x—y < 1, T), {£i,0 < x—y < 1, T), (£2, x—y = 0, T)}, {0}). 
Note that all markers are T since the guard on y faithfully represented the ones on x. 

From state vi, if Spoiler chooses the move < y < l,a, it is not obvious to which 
transitions in A this corresponds, and we thus explain in details how to compute the successor 
state. First observe that the only configuration in vi from which an a action is possible 
is the first one, with location Iq. In this configuration, the relation is < x — y < 1. 
Let us now explain what guard over x is induced by the relation C = 0<x — y<l 
and the region r' = < y < 1. Figure \^ illustrates this computation. The dotted area 




Figure 9: Construction of the induced guard. 



represents the set of the valuations over {x, y} satisfying the guard r' = < y < 1 and 



18 



BERTRAND, JERON, STAINER, AND KRICHEN 



the dashed area represents the relation C = < x — y < 1. The induced guard [r' f] C']|{a;} 
(i.e. the guard over x encoded by the guard r' on y through the relation C) is then the 
projection over clock x of the intersection of these two areas. In this example, the induced 
guard is < x < 2. Therefore, the transitions of A corresponding to the choice of Spoiler 
< y < l,a are as before the three ones originating in Iq, but this time they are over- 
approximated. Indeed, the induced guard [r' n C']|{a;} is not included in the original guard 
< X < 1 in A, i.e. a priori r' encodes more values than g. As a consequence, all the 
configurations in Spoiler's successor state are marked _L. Last, let us detail how the new 
relations are computed. Assuming Determinizator chooses not to reset y leads to state V2, 
in which for the configuration with location Iq, the relation is the smallest one containing 
{0 < X — y < 1) n (0 < y < 1) n (0 < X < 1), namely < x — y < 1. The relation for 

the last configuration in V2 is ((0 <x — y<l)n(0<y<l)n(0<x< 1))[^^q]^, which is 

same as {x = < y < l]^ , namely —l<x — y<0. 

As explained earlier, a strategy for Determinizator chooses in each state of V/j a set 
Y C X® of clocks to reset. With every strategy 11 for Determinizator we associate the 
TA B = Aut(n) obtained by merging a transition of Spoiler with the transition chosen by 
Determinizator just after. The following theorem links strategies of Determinizator with 
deterministic over-approximations of the original traces language and enlightens the interest 
of the game: 

Theorem 3.2 (|7]). Let A be a TA, and k, G N. For any strategy li of Determinizator 
in ^^^(fc,Af8); B = Aut(n) is a deterministic timed automaton over resources {k,M'^) and 
satisfies Traces(^) C Traces(;B). Moreover, ifU is winning, then Traces(^) = Traces(;B). 

When there is no winning strategy, one can either try to increase resources (number 
of clocks and/or maximal constant), or try to choose the best losing strategy, which is a 
concern. Indeed, the language inclusion seems to be a good criterion to compare two losing 
strategies, but it is not a total ordering. Alternatively, one can use the natural heuristics 
which tends to lose as late as possible (see [6]). In particular, for a game with k clocks 
and same maximal constant as the original timed automaton, there is a strategy which 
ensures not to lose before k moves (of each players): by choosing to reset a new clock 
at each of its moves, Determinizator ensures to perfectly encode all clocks of the original 
timed automaton. Other alternatives would be to consider heuristics based on quantitative 
measures over languages. 

3.2. Extensions to TAIOs and adaptation to tioco. In the context of model-based 
testing, the above-mentioned determinization technique must be adapted to TAIOs, as 
detailed in ||6j, and summarized below. The model of TAIOs is an expressive model of timed 
automata incorporating internal actions and invariants. Moreover, inputs and outputs must 
be treated differently in order to build from a TAIO A a DTAIO B such that A ^ B, and 
then to preserve tioco. 

• Internal actions are naturally part of the specification model. They cannot be 
observed during test executions and should thus be removed during determinization. 
In order to do so, a closure by internal actions is performed for each state during the 
construction of the game, that is, in each state, all the configurations reachable by 
internal actions are added to the set of configurations. To this attempt, states of the 
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game have to be extended since internal actions might be enabled from a subset of 
time-successors of the region associated with the state. Therefore, each configuration 
is associated with a proper region which is a time-successor of the initial region of 
the state. The closure by internal actions is effectively computed the same way as 
successors in the original construction when Determinizator is not allowed to reset 
any clock. It is well known that timed automata with silent transitions are strictly 
more expressive than standard timed automata Therefore, our approximation 
can be coarse, but it performs as well as possible with its available clock information. 

• Invariants are classically used to model urgency in timed systems. Taking into 
account urgency of outputs is quite important, indeed without the ability to express 
it, for instance, any dummy system would conform to all specifications. Ignoring all 
invariants in the approximation as done in [18j surely yields an io-abstraction: delays 
(considered as outputs) are over-approximated. In order to be more precise, while 
preserving the io-abstraction relation ^, with each state of the game is associated 
the most restrictive invariant containing invariants of all the configurations in the 
state. In the computation of the successors, invariants are treated as guards and 
their validity is verified at both ends of the transition. A state whose invariant is 
strictly over- approximated is treated as unsafe in the game. 

• Rather than over-approximating a given TAIO A, we aim here at building a DTAIO 
B io- abstracting A {A<B). Successors by outputs are over-approximated as in the 
original game, while successors by inputs must be under-approximated. The over- 
approximated closure by silent transitions is not suitable to under-approximation. 
Therefore, states of the game are extended to contain both over- approximated and 
under- approximated closures. Thus, the unsafe successors by an input (where pos- 
sibly an over-approximation would occur), are not built. 

Example 3.3. Figure represents a non- deterministic timed automaton A! that has in- 
variants and internal actions. It is a sub-automaton of the timed automaton we use in 



the next section (see Figure 13) to illustrate the approximate determinization for our test 
selection. 

X = \. T, {.t} 




Figure 10: Non- deterministic timed automaton A' (with invariants and internal actions). 



Using this automaton A! , let us illustrate how the game construction is adapted to deal 
with internal actions and invariants, by detailing part of the game Ga',(i,2) represented in 
Figure 

A state of Spoiler in the game is a triple (S-, 5*+, (/, 6/)) where S- (resp. S+) is 
the under- approximated (resp. over-approximated) closure by unobservable actions of the 
successors by some observable action, I is the invariant and bj is the marker which indicates 
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Figure 11: Part of the game ^^',(1,2)- 

a risk of approximation of the invariant. The invariant and the marker of Spoiler's states 
are written below the states. 

In the initial state of the game, {io, x—y = 0, T, {0}) E S- C. S.^.. Moreover, an internal 
action r can be fired for x = 1 along two different edges, which add two configurations, 
associated with the region y = 1 (because x—y = in the first configuration). Determinizator 
cannot reset y along an internal action, hence the relation for the configuration with location 
£5 is x — y = —1. Note that the region y = 1 is associated with the two last configurations in 
the initial state, reflecting that the internal action fired and thus the least value for y is 1. 
Also in this case, the closure (by internal actions) is not approximated, hence S- =5"+. On 
the other hand, it may be surprising that the invariant of this initial state is true whereas 
the invariant of the initial state of A' is x < 1. In fact, the invariant of a state is the 
smallest invariant containing the union, over all its configurations, of induced invariants. 
On this example, after an internal action from £0, delays are not constrained anymore in 
£1 and ie (invariants are true^. Thus the invariant in the initial state of the game is not 
approximated, so its marker is T. 

From this initial state. Spoiler can choose the regions y = lorl<y<2 together 
with action a?. For y = 1, this can only happen from the configuration with location £5. 
Indeed, the relation x — y = and the guard y = 1 induce a guard x = 1 which is not 
compatible with the outgoing edge from ii in A' . The computation of the successor state, 
e.g. when Determinizator chooses to reset y, is simple: no internal action is fireable and 
the invariant in £q is precisely expressed by y = 0. The situation is more complex when 
Spoiler chooses the region 1 < y < 2: in this case there are two successors by the observable 
action a? (leading to locations £% and £2), and for the first one internal actions may follow. 
We thus have to compute the closure by internal actions of the successor configuration 
by observable action a?. Before computing the closure, and assuming that Determinizator 
resets clock y, the successor state is composed of two configurations: {£2, x — y = 0,T) and 
{£e, X — y = 0,T) together with region y = 0. Along the r-loop on location £2, x is reset in 
A' whereas y cannot be reset in the game (because it is an internal action). Starting from 
configuration {£2,x — y = 0,T , {0}) and performing once the internal action t, the resulting 
configuration is thus {£2,x — y = — 1,T,{1}). This computation is iterated to obtain the 
closure by internal actions, which in such a case, will depend on the maximal constant (here 
2). Indeed, after {£2,x — y = — 1,T, {1}), the next configuration is {i2,x — y = — 2,T, {2}) 
and starting from {i2,x — y = — 2,T,{2}) the effect of one internal action would yield to 
(£2,x — y = — 3,T,{3}). However, x — y = —3 cannot be expressed in Re\2{{x,y}), so 
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it is approximated by the least relation of Re\2{{x,y}) containing it, that is x — y < —2. 
Similarly, region y = 3 is approximated by y > 2. As a consequence, the configuration 
{i2,x — y = — 3,T,{3}) is approximated by [t^^x — y< — 2, _L, (2, oo)) in 5"+. Note that 
this latter configuration is in Sj^ \ S- and thus separated from configurations in by 
two horizontal lines on Figure 



11 



Moreover, taking the union of all the invariants, we 
obtain true as invariant for this state, but since it is approximated for the last configuration 
[liiX — y < —2, _L, (2, oo)), its marker is _L. 

All in all, these modifications allow to deal with the full TAIO model with invariants, 
internal transitions and inputs/outputs. In particular, the treatment of invariants is con- 
sistent with the io-abstraction: delays are considered as outputs, thus over- approximated. 



Figure 14 represents a part of this game for the TAIO of Figure 13 The new game then 
enjoys the following nice property: 

Proposition 3.4 ([6]). Let A be a TAIO, and k,M'^ G N. For any strategy U of De- 
terminizator in the game GA,{k,M'S)) ^ = Aut(n) is a DTAIO over resources {k,M^) with 
B. Moreover, i/II is winning, then Traces(^) = Traces(i3). 

In other words, the approximations produced by our method are deterministic io- 
abstractions of the initial specification, hence the approximate determinization preserves 



tioco (Proposition 2.9), and conversely, sound test cases of the approximate determiniza- 



tion remain sound for the original specification (Corollary 2.10). Note that the proof of 



proposition 3.4 in [6] considers a stronger refinement relation, thus implies the same result 



for the present refinement relation. In comparison with our method, the algorithm proposed 
in [18j always performs an over-approximation, and thus preserves tioco only if the specifi- 
cation is input-complete; moreover all invariants are set to true in the resulting automata, 
so the construction does not preserve urgency. 



Complexity. The number of regions (resp. relations) over a set of clocks is exponential 
in the number of clocks. Thus, the number of possible configurations in the game is at 
most exponential in the cardinality of X U y and linear in the number of locations in 
A. As a consequence, the size of the game {i.e., number of states in the arena) is at 
most doubly exponential in |X U y| and exponential in \L-^\. In particular this bound also 
holds for the size of the generated deterministic TAIO, for every memoryless strategy of 
Determinizator. The overall complexity of this io-abstracting determinization algorithm is 
thus doubly exponential in the size of the instance (original TAIO and resources). 



4. Off-line test case generation 

In this section, we describe the off-line generation of test cases from timed automata speci- 
fications and test purposes. We first define test purposes, their role in test generation and 
their formalization as OTAIOs. We then detail the process of off-line test selection guided 
by test purposes, which uses the approximate determinization just defined. We also prove 
properties of generated test cases with respect to conformance and test purposes. 



22 



BERTRAND, JERON, STAINER, AND KRICHEN 



4.1. Test purposes. In testing practice, especially when test cases are generated manually, 
each test case has a particular objective, informally described by a sentence called test 
purpose. In formal test generation, test purposes should be formal models interpreted as 
means to select behaviors to be tested, either focusing on usual behaviors, or on suspected 
errors in implementations [13], thus typically reachability properties. They complement 
other selection mechanisms such as coverage methods [26j which, contrary to test purposes, 
are most often based on syntactical criteria rather than semantic aspects. Moreover, the 
set of goals covering a given criterion {e.g. states, transitions, etc) may be translated into a 
set of test purposes, each test purpose focusing on one such goal. 

As test purposes are selectors of behaviors, a natural way to formalize them is to use a 
logical formula characterizing a set of behaviors or an automaton accepting those behaviors. 
In this work we choose to describe test purposes as OTAIOs equipped with accepting states. 
The motivation is to use a model close to the specification model, easing the description 
of targeted specification behaviors. The following definition formalizes test purposes, and 
some alternatives are discussed in Section [5j 

Definition 4.1 (Test purpose). Let A = (L-^J^, J^f, ^f, S^, X^, 0, M-^, /-^, E-^) be a TAIO 
specification. A test purpose for ^ is a pair {TV, Accept ■'''') where: 

m TV = {L^^,i^^,J:f,^f,J:^,X^^,X^^,M^^,r^,E^^) is a complete OTAIO (in 
particular I'^^{i) = true for any i £ L'^'^) with Xj'' = X^ {TV observes proper 
clocks of A) and X^^ n = 0, 
• Accept^'' C L'^'^ is a subset of trap locations. 

In the following, we will sometimes abuse notations and use TV instead of the pair 
(T7^, Accept^''). During the test generation process, test purposes are synchronized with 
the specification, and together with their Accept locations, they will play the role of accep- 
tors of timed behaviors. They are non- intrusive in order not to constrain behaviors of the 
specification. This explains why they are complete, thus allowing all actions in all locations, 
and are not constrained by invariants. They observe behaviors of specifications by synchro- 
nizing with their actions (inputs, outputs and internal actions) and their proper clocks (by 



the definition of the product (Definition 1.5 ), observed clocks of TV are proper clocks of A, 
which mean that TV does not reset those clocks). However, in order to add some flexibility 
in the description of timed behaviors, they may have their own proper clocks. 




Figure 12: Test purpose TV. 



Example 4.2. Figure 12 represents a test purpose TV for the specification A of Figure^ 



This one has no proper clock and observes the unique clock x of A. It accepts sequences 
where r occurs at x = 1, followed by an input a at x < 1 (thus focusing on the lower 
branch of A where x is reset), and two subsequent b's. The label othw (for otherwise) on 
a transition is an abbreviation for the complement of specified transitions leaving the same 
location. For example in location £\, othw stands for {(true,r), (true, 6!), (x > l,a?)}. 
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4.2. Principle of test generation. Given a specification TAIO A and a test purpose 
(T7^, Accept^''), tlie aim is to build a sound and, if possible strict test case (TC, Verdicts) 
focusing on behaviors accepted by TV- As TV accepts sequences of A, but test cases 
observe timed traces, the intention is that TC should deliver Pass verdicts on traces of 
sequences of A accepted by TV in Accept^''. This property is formalized by the following 
definition: 

Definition 4.3. A test suite TS for A and TV is said to be precise if for any test case TC 
in TS, for any timed observation a in Traces(T"C), Verdict((T, TC) = Pass if and only if 

a G Traces(Seq(^t^^^^'''^°^^^) n Seqj^^^^^^Tv{TV)). 

Let A = {L^, i^, S^, ^f, S^, X^, 0, M^, I^,E-*) be the specification TAIO, and TV = 
(L^^,£j^,E^,E|^,S^,Xj^^Xj^,M^^,I^^,£;^^) be a test purpose for A, with its set 
Accept of accepting locations. The generation of a test case TC from A and TV proceeds 
in several steps. First, sequences of A accepted by TV are identified by the computation of 
the product V of those OTAIOs. Then a determinization step is necessary to characterize 
conformant traces as well as traces of accepted sequences. Then the resulting determinis- 
tic TAIO "DV is transformed into a test case TAIO TC' with verdicts assigned to states. 
Finally, the test case TC is obtained by a selection step which tries to avoid some Inconc 
verdicts. The different steps of the test generation process from A and TV are detailed in 
the following paragraphs. 



Computation of the product: First, the product V = A x TV is built (see Definition 1.5 
for the definition of the product), associated with the set of marked locations Accept^^ = 

X Accept^^. Let P = {L^ , , , r , E^). As XJ^ = X^, we get 

= and X^ = X^U X^^ , thus V is in fact a TAIO. 

The effect of the product is to unfold A and to mark locations of the product by 
Accept'', so that sequences of A accepted by TV are identified. As TV is complete, 
Seq(T'P) ixTv= (M>o x (Y,'^'^ x 2'^°'^))* , thus, by the properties of the product (see equa- 
tion 1.2), Seq('P) ixj^'P^ Seq(^) i.e. the sequences of the product after removing resets of 
proper clocks of TV are the sequences of A. As a consequence Traces('P) = Traces(^), 
which entails that V and A define the same sets of conformant implementations. 



Considering accepted sequences of the product V, by equation |1.3| we get the equality 
Seq^^j.gp^p (P) = Seqi^At^^p^ '^°^^)r\Seqf^^^^p^Tv{TV), which induces the desired characteri- 
zation of accepted traces: Traces^^^gp^p (P) = Traces(Seq(.At*-"^p''''^°'''^) n Seq^,_^gp^rT'(TP)). 

Using the notation pref{T) for the set of prefixes of traces in a set of traces T, we note 
RTraces(^, TP) = Traces(^) \ pref (Traces /^^^^^^v {V)) for the set of traces of A which are 
not prefixes of accepted traces of V. In the sequel, the principle of test selection will be to 
try to select traces in Traces^^^gp^T'(T') (and assign to them the Pass verdict) and to try to 
avoid or at least detect (with an Inconc verdict) those traces in RTraces(^, T7^), as these 
traces cannot be prefixes of traces of sequences satisfying the test purpose. 



Example 4.4. Figure 13 represents the product V for the specification A in Figure^ and 



the test purpose TV in Figure As TV describes one branch of A, the product is very 
simple in this case, e.g. intersection of guards are trivial. The only difference with A is the 
tagging with Accept''. 
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Figure 13: Product V = AxTV. 



Approximate determinization of V into T>V : We now want to transform V into a deter- 



ministic TAIO T>V such that V ^ VP, which by Proposition 2.9) wih entail that im- 
plementations conformant to V (thus to A) are still conformant to W. If V is already 
deterministic, we simply take VP = P. Otherwise, the approximate determinization of 
Section [3] provides a solution. The user fixes some resources (A;,M^''), then a determin- 
istic io-abstraction VP of P with resources {k,M^'^) is computed. By Proposition 3.4 
we thus get that VP io-abstracts P. VP is equipped with the set of marked locations 
Accept^'' consisting of locations in L'^''^ containing some configuration whose location is in 
Accept''. As a consequence traces of VP which are traces of sequences accepted by P in 
Accept'' are accepted by VP in Accept^^, formally Jraces{VP) n Traces(Seq^^^gp^p(P)) = 
Traces(PP)nTraces^j_^gp^-p(P) C Traces ^^^^^^^^-dv (VP). This means that extra accepted traces 
may be added due to over- approximations, some traces may be lost (including accepted ones) 
by under-approximations, but if the under-approximation preserves some traces that are ac- 
cepted in P, these are still accepted in VP. If the determinization is exact (or P is already 
deterministic), of course we get more precise relations between the traces and accepted traces 
of P and VP, namely Traces(DP) = Traces(T') and Traces f^^^^^^-Dv (VP) = Traces^^j_gp^7:'(P). 



Example 4.5. Figure I4 partially represents the game ^-p,(i,2) for the TAIO P of Figure 13 
where, for readability reasons, some behaviors not co-reachable from Accept^'' ( dotted green 
states) are omitted. Notice that the construction of the initial part of the game was explained 
in Example 3.3. A strategy 11 for Determinizator is represented by bold arrows. 11 is not 



winning (the unsafe configuration, in gray, is unavoidable from the initial state), and in 
fact an approximation is performed. VP, represented in Figure [75| is simply obtained from 
Gv,(i,2) '^'Ti'd the strategy H by merging transitions of Spoiler and those of Determinizator in 
the strategy. 



Generating TC' from VP: The next step consists in building a test case (TC, Verdicts) 
from VP. The main point is the computation of verdicts. Pass verdicts are simply defined 
from Accept^''. Fail verdicts that should detect unexpected outputs and delays, rely on 
a complementation. The difficult part is the computation of Inconc states which should 
detect when Accept^'' is not reachable (or equivalently None states, those states where 
Accept^'' is still reachable) and thus relies on an analysis of the co-reachability to locations 
Accept'"'. Another interesting point is the treatment of invariants. First TC will have no 
invariants (which ensures that it is non-blocking). Second, invariants in VP are shifted to 
guards in TC and in the definition of Fail so that test cases check that the urgency specified 
in A is satisfied by X. 
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Figure 14: Game Q'p,{i,2)- 
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Figure 15: Deterministic automaton W = Aut{U). 



The test case constructed from VV = (L^^, ^g"^, Sf ^, Sf^, 0, X^^, 0, M^^, E 
and Accept^'' is the pair (TC, Verdicts) where: 

• TC = (L^'^',^5'c',Sfc',sp',0,XjJ'c',0,M^c'^ jrc'^^rC) ^he TAIO such that: 

— L^*^' = L^^ U {^Faii} where is a new location; 



Iq^ = is the initial location; 
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YJ^^' = Yjf' = Yif and S/''^' = S^"' = S^, i.e. input/output alphabets are 
mirrored in order to reflect the opposite role of actions in the synchronization 
of TC and 1- 



XJ^' = X^^ and Xjc' = X^^ 



- = true for any i e L'^^' ; 

- E^^' = Ef^ U Ee^^.^ where 

Ef^ = {{i,g Al^^{£),a,X',i') \ {£, g,a, X' ,£') G S^^} and 



Ei^^,, = Ui,gM^^{i),a,X;' ,1 



and 5 = ^ \l l^i^g^a,X' ,v)(iET^T 9 



• Verdicts is the partition of S''^^ defined as follows: 
-Pass = U,eAccept-^(Wx/-^W), 

- None = coreach(P'P, Pass) \ Pass, 

- Fail = {4aii} X R^r' U {{£, ^I^^mi G L^^}; 

- Inconc = 5^^ \ (Pass U Fail U None), 

The important points to understand in the construction of TC are the completion to 
Fail and the computation of None, which, together with Pass, define Inconc by comple- 
mentation. 

For the completion to Fail, the idea is to detect unspecified outputs and delays with 
respect to W. Remember that outputs of W are inputs of TC . Moreover, authorized 
delays in W are defined by invariants, but remember that test cases have no invariants 
(they are true in all locations). First, all states in (^, ^I^^ {£)),£ G L'^^ , i.e. states where the 
invariant runs out, are put into Fail which reflects the counterpart in TC of the urgency 
in DV. Then, in each location £, the invariant I^^{£) in DV is removed and shifted to 
guards of all transitions leaving £ in TC, as defined in Ef^. Second, in any location £, 
for each input a G Sf*^' = Sf'', a transition leading to ^paii is added, labeled with a, and 
whose guard is the conjunction of I{£) with the negation of the disjunction of all guards 
of transitions labeled by a and leaving £ (thus true if no a-action leaves £), as defined in 
E^felu- is then easy to see that TC is input-complete in all states. 

The computation of None is based on an analysis of the co-reachability to Pass. 
None contains all states co-reachable from locations in Pass. Notice that the set of states 
coreach(D'P, Pass), and thus None, can be computed symbolically as usual in the region 
graph of DV, or more efficiently using zones. 



Example 4.6. Figure 16 represents the test case TC obtained from T>V . For readability 
reasons, we did not represent transitions in Ei^_^^^, except the one leaving P\. In fact these 
are removed in the next selection phase as they are only fireable from states where a verdict 
has already been issued. The rectangles attached to locations represent the verdicts in these 
locations when clock y progresses between and 2, and after 2: dotted green for Pass, black 
for None, blue grid for Inconc and crosshatched red for Fail. For example, in £"2, the 
verdict is initially None, becomes Inconc if no b is received immediately, and even Fail 
if no b is received before one time unit. Notice that in order to reach a Pass verdict, one 
should initially send a after one and strictly before two time units, and expect to receive two 
consecutive b's immediately after. 
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Figure 16: Test case TC with verdicts 



Selection ofTC: So far, the construction of TC' determines Verdicts, but does not perform 
any selection of behaviors. A last step consists in trying to control the behavior of TC' in 
order to avoid Inconc states (thus stay in pref {TraceSf^^^^^^v {V))), because reaching Inconc 
means that Pass is unreachable, thus TV cannot be satisfied anymore. To this aim, guards 
of transitions of TC' are refined in the final test case TC in two complementary ways. 
First, transitions leaving a verdict state (Fail, Inconc or Pass) are useless, because the 
test case execution stops when a verdict is issued. Thus for each transition, the guard is 
intersected with the predicate characterizing the set of valuations associated with None 
in the source location. This does not change the verdict of traces. Second, transitions 
arriving in Inconc states and carrying outputs can be avoided (outputs are controlled by 
the test case), thus for any transition labeled by an output, the guard is intersected with 
the predicate characterizing None and Pass states in the target location (i.e. states that 
are not in Inconc, as Fail cannot be reached by an output). The effect is to suppress 
some traces leading to Inconc states. All in all, traces in TC are exactly those of TC' that 
traverse only None states (except for the last state), and do not end in Inconc with an 
output. This selection does not impact on the properties of test suites (soundness, strictness, 
precision and exhaustiveness) as will be seen later. 



Example 4.7. Figure 11 represents the test case obtained after this selection phase. One 
can notice that locations f i2,^"i3 and i" 21, 22 have been removed since they can only 
be reached from Inconc states, thus a verdict will have been emitted before reaching those 
locations. The avoidance 0/ Inconc verdicts by outputs cannot be observed on this example. 
However, with a small modification of A consisting in adding initially the reception of an 
a before one time unit, and not followed by two b 's but e.g. one c, the resulting transition 
labeled with {0 < y < l,a!) in TC could be cut, producing the same TC. 

Remark 4.8. Notice that in the example, falling into Inconc in £"0 could be avoided by 
adding the invariant y < 2, with the effect of forcing to output a. More generally, invariants 
can be added to locations by rendering outputs urgent in order to avoid Inconc, while 
taking care of keeping test cases non-blocking, i. e. by ensuring that an output can be done 
just before the invariant becomes false. More precisely, is the projection of None on 
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Inconc = {£"„} X [2,00) U {f'j} x (0,1] U {t 1} x (0,oo) 
Fail = {C^,,,} X Rj„ U {^3, f '4} x (0, 00) U {£"2} x (1, 00) 



Figure 17: Final test case TC after selection 



i if Inconc is reachable by letting time elapse and it preserves the non-blocking property, 
true otherwise. 



Complexity. Let us discuss the complexity of the construction of TC from W. Note that 
the size of TAIO TC is linear in the size of W but the difficulty lies in the computation 
of Verdicts. Computing Pass is immediate. The set coreach(Pass) can be computed 
in polynomial time (more precisely in C'(|L^^|.|X^''|.|M^'''|)). To explain this, observe 
that guards in the TAIO T>V are regions and with each location i is associated an initial 
region such that guards of transitions leaving ^ are time successors of r^. Thus during 
the computation of coreach(Pass), for each location ^, one only needs to consider these 
C(|X^'''|.|M^''|) different regions in order to determine the latest time-successor r™^^ of 
ri which is co-reachable from Pass. Then None states with location i are exactly those 
within regions that are time-predecessors of r™**^. For the same reason (number of possible 
guards outgoing a given location) -E^p^;, can be computed in polynomial time. Last the Fail 
verdicts in locations (except for ^paii) are computed in linear time by complementing the 
invariants in T>V. The test selection can be done by inspecting all transitions: a transition 
is removed if either the source state is a verdict state, or it corresponds to an output action 
and the successor are Inconc states. This last step thus only requires linear time. To 
conclude, the overall complexity of construction of TC from W is polynomial. 



4.3. Test suite properties. We have presented the different steps for the generation of 
a TAIO test case from a TAIO specification and an OTAIO test purpose. The following 
results express their properties. 

Theorem 4.9. Any test case TC built by the procedure is sound for A. Moreover, ifW is 
an exact approximation ofV ('i.e. Traces ("DP) = Traces{V)), the test case TC is also strict 
and precise for A and TV . 

The proof is detailed below, but we first give some intuition. As a preamble, notice 
that, as explained in the paragraph on test selection, traces of TC' are not affected by the 
construction of TC. In particular, the transitions considered in the proof are identical in 
TC and TC' . Soundness comes from the construction of -E^p^;, in TC and preservation of 



soundness by the approximate determinization W of V given by Corollary 2.10 When 
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DV is an exact determinization of V, T>V and V have same traces, which also equal traces 
of A since TV is complete. Strictness then comes from the fact that DV and A have the 
same non-conformant traces, which are captured by the definition of -E'^p^ji in TC. Precision 
comes from Traces^^.^gp^oT' (PP) = Traces^^^gp^T'('P) and from the definition of Pass. 

When is not exact however, there is a risk that some behaviors allowed in DV are not 
in "P, thus some non-conformant behaviors are not detected, even if they are executed by TC. 
Similarly, some Pass verdicts may be produced for non-accepted or even non-conformant 
behaviors. However, if a trace in Traces^^.^.^p^p (P) is present in TC and observed during 
testing, a Pass verdict will be delivered. In other words, precision is not always satisfied. 



but the "only if" direction of precision (Definition 4.3) is satisfied. 



Proof. Soundness: To prove soundness, we need to show that for any I £ I{A) , X f ails TC 
implies -i(Xtioco^). 

Assuming that XfailsTC, there exists a trace a G Traces(X) n TraceSpaiiCT'C). By 
the construction of the set Fail in TC, there are two cases: either a leads to a location 
(£, in DV, or a leads to a state with location ^paii- In the first case, a = a' .6 
where a' G Traces(P7^) and 5 > Q violates the invariant in the location of W after a', 
and in the second case, by the construction of -E'^p^ji, cr = cr' .a where u' G Traces(2?'P) 
and a £ T,f^ is unspecified in DV after a' . In both cases, by definition, this means that 
-■(ItiocoPP), which proves that TC is sound for DV. Now, as DV is an io-abstraction of 



V {i.e. V :< DV), by Corollary 2.10 this entails that TC is sound for V. Finally, we have 
Traces(T') = Traces(^), which trivially implies that A ^V, and thus that TC is also sound 
for A. 

Strictness: For strictness, in the case where DV is an exact approximation of V, we 
need to prove that for any Z G I{A), -i(X||TC tioco^) implies that XfailsTC. Suppose 
that -i(X||TC tioco^). By definition, there exists a G Traces(^) and a G out{I\\TC after a) 
such that a ^ out{A after a). Since DV is an exact approximation of V, we have the equali- 
ties Traces(P'P) = Traces(P) = Traces(^), thus a G Traces(PT') and a ^ out{DV after a). 
By construction of Fail in TC, it follows that a. a G TracesFaii(T"C) which, together with 
a.a G Traces(X), implies that Xf ails TC Thus TC is strict. 

Precision: To prove precision, in the case of exact determinization, we have to show 
that for any trace a, Verdict((T, TC) = Pass <;=^> a G Traces(Seq^^^_gp^rp(T7-') n Seq(^)). 
The definition of Pass = U^eAccept'^'''(i^} ^ I'^'''{tj) in TC implies that a Pass verdict 
is produced for a exactly when a G Traces^^^gp^c-p (PT) which equals Traces^^j_gp^p (T) = 
Traces(Seq^^^gp^r-p(TT) n Seq(^)) when DV is exact. □ 



Example 4.10. The test case TC of Figure 11 comes from an approximate determinization. 



However, the approximation comes after reaching Inconc states. More precisely, in the gray 



state of the game in Figure 14 . the approximation starts in the time interval (2,oo). This 
state corresponds to location i in TC where the verdict is Inconc as soon as a non null 
delay is observed. The test case is thus strict and precise, despite the over- approximation 
in the determinization phase. 

In the following, we prove an exhaustiveness property of our test generation method 
when determinization is exact. For technical reasons, we need to restrict to a sub-class of 
TAIOs defined below. We discuss this restriction later. 
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Definition 4.11. We say that an OTAIO A is repeatedly observable if from any state of 
A, there is a future observable transition, i.e. Vs G S'^, there exists fx such that s A and 
Trace{n) ^ M>o. 

Theorem 4.12 (Exhaustiveness). Let A be a repeatedly observable TAIO which can be 
exactly determinized by our approach. Then the set of test cases that can be generated from 
A by our method is exhaustive. 

Proof Let A = {L^,£^,Ef,^f,J:^, X^,$, M-^, , E^) he the TAIO specification, and I = 
{L^,e^,T,f,'Ef,T,^,X^,<l), M^,P,E'^) any non-conformant implementation in I{A). The 
idea is now to prove that from A and I, one can build a test purpose TV such that the test 
case TC built from A and TV may detect this non-conformance, i.e. If ails TC. 

By definition of -i(X tioco A), there exists a G Traces(^) and a e T,f \J M>o such that 
a G outiX after cr) but a ^ out{A after a). Since A is repeatedly observable, there also 
exists S G R>o and b G "^obs such that a.S.b G Traces(^). 

As A can be determinized exactly by our approach, there must exist some resources 
{k, M) and a strategy 11 for Determinizator in the game Q_A.,ik,M) ^^^^ ^^^^ Traces(Aut(n)) = 
Traces(^). 

From the non-conformant implementation X, a test purpose (TP, Accept^'') can be 
built, with TV = {L^^, E^, Sj^, S^, XJ^, XJ"^, M^^, E^^), X^^ = XjU X'^^tW 
and XJ"^ = 0, and a.5.b G Traces^^^^gp^rp but none of its prefixes is in Traces^^^^gp^r-p . The 
construction of TV relies on the region graph of X|| Aut(n). First a TAIO TV' is built which 
recognizes exactly the traces read along the path corresponding to a in the region graph of 
I||Aut(n), followed by a transition b with the guard corresponding to the one in Aut(n). 
In particular it recognizes the trace a.d.b. The test purpose (TP, Accept^^) is then built 
such that TV accepts in its states Accept the traces of TV' . Note that TV should be 
complete for S, thus locations of TV' should be completed by adding loops without resets 
for all actions in S^, and adding, for all observable actions, transitions to a trap location 
guarded with negations of their guards in TV' . 

Now consider our test generation method applied to TV and A. First V = Ax TV is 
built, and we consider the game GA,{k',M') with k' = k + \Xp'^\ and M' = max(M, M''"''). 
One can then define a strategy 11' composed of the strategy 11 for the k first clocks, and 
following the resets of TV (which is deterministic) for the other clocks corresponding to 
those in Xp'^ . The construction of (PT, Accept^^) following the strategy 11' thus ensures 
that Traces(PT) = Traces(P) and Traces^^^gp^cp (DT) = Traces^^^gp^7:'('P). 

Finally, let TC be the test case built from W. Observe that TC after a.S.b C Pass, 
but TC after a.S ^ Pass. As a consequence, TC after a C None. Moreover we have 
a ^ out{A after a), hence a.a G TracesFaii(TC) and as a.a G Traces(X), we can conclude 
that X fails TC. □ 



Discussion: The hypothesis that A is repeatedly observable is in fact not restrictive for a 
TAIO that is dctcrminizablc by our approach. Indeed, such a TAIO can be transformed into 
a repeatedly observable one with same conformant implementations, by first dctcrminizing 
it, and then completing it as follows. In all locations, a transition labeled by an input is 
added, which goes to a trap state looping for all outputs, and is guarded by the negation 
of the union of guards of transitions for this input in the deterministic automaton. 



OFF-LINE TEST SELECTION WITH TEST PURPOSES FOR NON-DET. TIMED AUTOMATA 31 



When A cannot be determinized exactly, the risk is that some non-conformance may 
be undetectable. However, the theorem can be generalized to non-determinizable automata 
with no resets on internal action. Indeed, in this case, in the game with resources {k,M), 
where k is the length of the finite non-conformant trace a. a, the strategy consisting in 
resetting a new clock at each observable action allows to remain exact until the observation 
of non- conformance (see remark after Theorem 3.2). The proof of theorem 4.12 can be 
adapted using this strategy. 



5. Discussion and related work 



Alternative definitions of test purposes. The definition of test purposes depends on 
the semantic level at which behaviors to be tested are described {e.g. sequences, traces). 
This induces a trade-off between the precision of the description of behaviors, and the 
cost of producing test suites. In this work, test purposes recognize timed sequences of the 
specification A, by a synchronization with actions and observed clocks. They also have 
their own proper clocks for additional precision. The advantage is a fine tuning of selection. 
The price to be paid is that, for each test purpose, the whole sequence of operations, 
including determinization which may be costly, must be done. An alternative is to define 
test purposes recognizing timed traces rather than timed sequences. In this case, selection 
should be performed on a deterministic io-abstraction B ol A obtained by an approximate 
determinization of A. Then, test purposes should not refer to .4's clocks as these are lost 
by the approximate determinization. Test purposes should then either observe B's clocks, 
and thus be defined after determinization, or use only proper clocks in order not to depend 
on B, at the price of further restricting the expressive power of test purposes. In both 
cases, test purposes should preferably be deterministic in order to avoid a supplementary 
determinization after the product with B. The main advantage of these approaches is that 
the specification is determinized only once, which reduces the cost of producing a test suite. 
However, the expressive power of test purposes is reduced. 

Test execution. Once test cases are selected, it remains to execute them on a real imple- 
mentation. As a test case is a TAIO, and not a simple timed trace, a number of decisions 
still need to be taken at each state of the test case: (1) whether to wait for a certain delay, 
or to receive an input or to send an output (2) which output to send, in case there is a 
choice. It is clear that different choices may lead to different behaviors and verdicts. Some 
of these choices can be made either randomly {e.g. choosing a random time delay, choosing 
between outputs, etc), or can be pre-established according to user-defined strategies. One 
such policy is to apply a technique similar to the control approach of [10] whose goal is to 
avoid RTraces(^, TP). 

Moreover, the tester's time observation capabilities are limited in practice: testers only 
dispose of a finite-precision digital clock (a counter) and cannot distinguish among obser- 
vations which elude their clock precision. Our framework may take this limitation into 
account. In [18j assumptions on the tester's digital clock are explicitly modeled as a special 
TAIO called Tick, synchronized with the specification before test generation, then rely- 
ing to the untimed case. We could imagine to use such a Tick automaton differently, by 
synchronizing it with the resulting test case after generation. 
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Related work. As mentioned in the introduction, off-line test selection is in general re- 
stricted to deterministic automata or known classes of determinizable timed automata. An 
exception is the work of fTE\ which relies on an over-approximate determinization. Com- 
pared to this work, our approximate determinization is more precise (it is exact in more 
cases), it copes with outputs and inputs using over- and under-approximations, and pre- 
serves urgency in test much as possible. Another exception is the work of [lOj . 
where the authors propose a game approach whose effect can be understood as a way to 
completely avoid RTraces(^, TP), with the possible risk of missing some or even all traces 
in ]9re/(Traces^^^gp^-p ("P)). Our selection, which allows to lose this game and produce an 
Inconc verdict when this happens, is both more liberal and closer to usual practice. 

In several related works [161 E]) test purposes are used for test case selection from 
TAIOs. In all these works, test purposes only have proper clocks, thus cannot observe 
clocks of the specification. 

It should be noticed that selection by test purposes can be used for test selection with 
respect to coverage criteria [26j. Those coverage criteria define a set of elements (generally 
syntactic ones) to be covered {e.g. locations, transitions, branches, etc). Each element can 
then be translated into a test purpose, the produced test suite covering the given criteria. 

6. Conclusion 

In this article, we presented a complete formalization and operations for the automatic off- 
line generation of test cases from non-deterministic timed automata with inputs and outputs 
(TAIOs). The model of TAIOs is general enough to take into account non-determinism, 
partial observation and urgency. One main contribution is the ability to tackle any TAIO, 
thanks to an original approximate determinization algorithm. Another main contribution 
is the selection of test cases with expressive test purposes described as OTAIOs having 
the ability to precisely select behaviors to be tested based on clocks and actions of the 
specification as well as proper clocks. Test cases are generated as TAIOs using a symbolic 
co-reachability analysis of the observable behaviors of the specification guided by the test 
purpose. 

A first perspective of this work is to implement the approach in a test generation tool. 
Currently, the approximate determinization has been prototyped in Python thanks to a 
binding of the UPPAAL DBM library [25]. Other perspectives could be to combine this 
approach with the one of [Hj for models with data, for the generation of test cases from 
models with both time and data in the spirit of [3j, but generalized to non-deterministic 
models. 

Acknowledgements: we would like to thank the reviewers for their constructive comments 
that allowed us to improve this article. 
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